Search for: All records

Robust Markov Decision Processes (MDPs) offer a promising framework for computing reliable policies under model uncertainty. While policy gradient methods have gained increasing popularity in robust discounted MDPs, their application to the average-reward criterion remains largely unexplored. This paper proposes a Robust Projected Policy Gradient (RP2G), the first generic policy gradient method for robust average-reward MDPs (RAMDPs) that is applicable beyond the typical rectangularity assumption on transition ambiguity. In contrast to existing robust policy gradient algorithms, RP2G incorporates an adaptive decreasing tolerance mechanism for efficient policy updates at each iteration. We also present a comprehensive convergence analysis of RP2G for solving ergodic tabular RAMDPs. Furthermore, we establish the first study of the inner worst-case transition evaluation problem in RAMDPs, proposing two gradient-based algorithms tailored for rectangular and general ambiguity sets, each with provable convergence guarantees. Numerical experiments confirm the global convergence of our new algorithm and demonstrate its superior performance. 

Abstract Artificial intelligence (AI) has gained widespread public interest in recent years. However, as AI literacy remained excluded from the standard academic curricula, AI education in the US was predominantly offered through extra-curricular activities, which limited AI learning exposure to only a select group of students. Given these limitations, the need to integrate AI literacy education into the standard curricula is increasingly evident. This study investigated the integration of AI learning in an advanced biology course. Thirty-seven students participated in four lessons embedding AI learning in biology contexts. The interplay of students’ AI learning and biology knowledge was examined from the quantitative measure of conceptual understanding and qualitative analysis of interdisciplinary reasoning. This concurrent triangulation research design utilized results from both quantitative and qualitative analyses to develop a comprehensive understanding of students’ AI learning in the biology context. The results of the study showed a significant improvement in students’ AI concepts. Students’ biology knowledge had a slight increase, but it was not statistically significant. Both quantitative and qualitative results underscored a close connection between students’ AI learning and their biology knowledge, though the quantitative findings were not conclusive in some lessons. The article concluded with a discussion of the potential reasons for those discrepancies. In addition, suggestions were provided for future research and practitioners who are interested in integrating AI education across curricula.